Probability, programs, and the mind: Building structured Bayesian models of cognition

Human thought is remarkably flexible: we can think about infinitely many different situations despite uncertainty and novelty. Probabilistic models of cognition (Chater, Tenenbaum, & Yuille, 2006) have been successful at explaining a wide variety of learning and reasoning under uncertainty. They have borrowed tools from statistics and machine learning to explain phenomena from perception (Yuille & Kersten, 2006) to language (Chater & Manning, 2006). Traditional symbolic models (e.g. Newell, Shaw, & Simon, 1958; Anderson & Lebiere, 1998), by contrast, excel at explaining the productivity of thought, which follows from compositionality of symbolic representations. Indeed, there has been a gradual move toward more structured probabilistic models (Tenenbaum, Kemp, Griffiths, & Goodman, 2011) that incorporate aspects of symbolic methods into probabilistic modeling. Unfortunately this movement has resulted in a complex “zoo” of Bayesian models. We have recently introduced the idea that using programs, and particularly probabilistic programs, as the representational substrate for probabilistic modeling tames this unruly zoo, fully unifies probabilistic with symbolic approaches, and opens new possibilities in cognitive modeling. The goal of this tutorial is to introduce probabilistic models of cognition from the point of view of probabilistic programming, both as a unifying idea for cognitive modeling and as a practical tool. The probabilistic programming language Church (Goodman, Mansinghka, Roy, Bonawitz, & Tenenbaum, 2008), mathematically grounded on the stochastic λ-calculus, provides a universal language for representing probabilistic models. We will use Church to introduce key ideas and examples of probabilistic modeling. A Church program represents a probabilistic model, and hence inferences that can be drawn from this model, without committing to a process level implementation of inference. This will allow us to focus the tutorial on structured representations and probabilistic inference phenomena without worrying about the details of inference algorithms (such as Markov chain Monte Carlo) that tutorials on Bayesian modeling often become bogged down in. On the other hand, because there are existing inference tools for Church (e.g. Wingate, Stuhlmüller, & Goodman, 2011), students will get hands-on experience with performing inference over different probabilistic models. The tutorial will include several in-depth case studies where the probabilistic programming viewpoint is particularly useful. After introducing the basic phenomena of probabilistic reasoning—explaining away, screening off, etc—we will turn to the representation of intuitive theories and the connection between probabilistic programs, intuitive theories, and mental simulation. We will focus in particular on folk physics and folk psychology, showing that they can be captured as probabilistic programs, that this explains data from human experiments, and that they can be productively integrated together.